Data center exterior wall penetrating air handling technology

ABSTRACT

Described herein is an integrated data center that provides for efficient cooling, as well as efficient wire routing.

PRIORITY CLAIM

This application is a continuation of U.S. patent application Ser. No.15/663,169 entitled “Facility Including Externally Disposed Data CenterAir Handling Units” filed on Jul. 28, 2017, which is a continuation ofU.S. patent application Ser. No. 15/291,005 entitled “Facility IncludingExternally Disposed Data Center Air Handling Units” filed Oct. 11, 2016,which is a continuation of U.S. patent application Ser. No. 13/732,942entitled “Electronics Equipment Data Center or Co-Location FacilityDesigns and Methods of Making and Using the Same” filed Jan. 2, 2013,which is a continuation of U.S. patent application Ser. No. 12/384,109entitled “Electronic Equipment Data Center or Co-Location FacilityDesigns and Methods of Making and Using the Same” filed Mar. 30, 2009,which claims priority to U.S. Provisional Patent Application No.61/040,636 entitled “Electronic Equipment Data Center or Co-LocationFacility Designs and Methods of Making and Using the Same,” filed onMar. 28, 2008, which application is expressly incorporated by referenceherein. This application is also a continuation of U.S. patentapplication Ser. No. 12/138,771 entitled “Electronic Equipment DataCenter or Co-location Facility Designs and Methods of Making and Usingthe Same” filed Jun. 13, 2008, which application claims priority to U.S.Provisional Patent Application No. 60/944,082 entitled “ElectronicEquipment Data Center or Co-location Facility Designs and Methods ofMaking and Using the Same” filed Jun. 14, 2007, which applications areexpressly incorporated by reference herein.

BACKGROUND Field of the Invention

The present invention relates to electronic equipment data center orco-location facility designs and methods of making and using the same inan environmentally aware manner.

Background of the Invention

Data centers and server co-location facilities are well-known. In suchfacilities, rows of electronics equipment, such as servers, typicallyowned by different entities, are stored. In many facilities, cabinetsare used in which different electronics equipment is stored, so thatonly the owners of that equipment, and potentially the facilityoperator, have access therein. In many instances, the owner of thefacilities manages the installation and removal of servers within thefacility, and is responsible for maintaining utility services that areneeded for the servers to operate properly. These utility servicestypically include providing electrical power for operation of theservers, providing telecommunications ports that allow the servers toconnect to transmission grids that are typically owned bytelecommunication carriers, and providing air-conditioning services thatmaintain temperatures in the facility at sufficiently low levels forreliable operation.

There are some well-known common aspects to the designs of thesefacilities. For example, it is known to have the electronic equipmentplaced into rows, and further to have parallel rows of equipmentconfigured back-to back so that each row of equipment generally forcesthe heat from the electronic equipment toward a similar area, known as ahot aisle, as that aisle generally contains warmer air that results fromthe forced heat from the electronics equipment. In the front of theequipment is thus established a cold aisle.

There are different systems for attempting to collect hot air thatresults from the electronics equipment, cooling that hot air, and thenintroducing cool air to the electronics equipment. Theseair-conditioning systems also must co-exist with power andcommunications wiring for the electronics equipment. Systems in whichthe electronics equipment is raised above the floor are well-known, asinstalling the communications wiring from below the electronicsequipment has been perceived to offer certain advantages. Routing wiringwithout raised floors is also known—though not with systematicseparation of power and data as described herein.

In the air conditioning units that are used in conventional facilitysystems, there are both an evaporator unit and a condenser unit. Theevaporator units are typically located inside a facility and thecondenser units are typically disposed outside of the facility. Theseunits, however, are not located in standardized, accessible andrelatively convenient positions relative to the facility should any ofthe units need to be accessed and/or removed for repair or replacement.Further, these units are not themselves created using an intentionallytransportable design.

SUMMARY

The present invention provides an integrated data center that providesfor efficient cooling, as well as efficient wire routing.

In one aspect is provided a facility with an internal area and anexternal area in an external environment for maintaining electronicequipment disposed in a plurality of cabinet clusters in the internalarea at a cool temperature, the facility comprises a building thatincludes an exterior load wall separating the internal area and theexternal area and a plurality of exterior wall openings in the exteriorload wall. A floor is within the internal area of the building on whichthe plurality of cabinet clusters are disposed and a plurality ofcabinets for holding the electronic equipment therein, the plurality ofcabinets are positioned in a plurality of rows within each of aplurality of cabinet clusters so that the electronic equipment disposedwithin the cabinets emit heated air from the cabinets in each row ofeach cabinet cluster toward a central hot air area associated with eachcabinet cluster. Also part of this embodiment is a plurality of supportbrackets within each cabinet cluster, disposed along each of theplurality of rows, that together provide support for distribution powerwiring and conduits, electronic equipment power wiring and conduits, andcommunication wiring. A portion of each of the support brackets isdisposed above the plurality of cabinets within each cabinet cluster,and some of the distribution power wiring and conduits string acrossother cabinets in other cabinet clusters. A thermal shield is supportedby the at least some of the plurality of support brackets, such that thethermal shield provides a contiguous wall around the central hot airarea and defines a hot air containment chamber that traps the heated airwithin the central hot air area and causes substantially all the heatedair within the central hot air area to rise up within the hot aircontainment chamber.

A plurality of air conditioning units are disposed in the external areaoutside the building that each receive heated air, emit cooled air, andemit vented air, wherein the vented air is released into the externalenvironment. A warm air escape gap, within the building, is disposedabove the hot air containment chamber, such that the warm air escapechannel feeds the heated air to the plurality of air conditioning units,the warm air escape gap being lowerly bounded by a false ceiling. Coolair ducts are within the building that couple the plurality of airconditioning units and the cold aisles, the cool air ducts beingdisposed below the false ceiling and delivering cool air from theplurality of air conditioning units toward the plurality of rows ofcabinets within each of the plurality of cabinet clusters. Warm airconnectors and cool air duct connectors that respectively connect thewarm air escape channel and the cold air ducts to the plurality of airconditioning units, which pass through the plurality of exterior wallopenings.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects and features of the present invention willbecome apparent to those of ordinary skill in the art upon review of thefollowing description of specific embodiments of the invention inconjunction with the accompanying figures, wherein:

FIG. 1A illustrates a floor design used in a data center or co-locationfacility according to the present invention.

FIG. 1B illustrates floor-based components disposed over the floordesign according to the present invention.

FIG. 1C illustrates a perspective cut-away view along line c-c from FIG.1(a) of FIG. 1(a) according to the present invention.

FIGS. 2A, 2B, 2C illustrate various cut-away perspective views of thethermal compartmentalization and cable and conduit routing systemaccording to the present invention.

FIGS. 3A and 3B illustrate modular thermal shields used in the thermalcompartmentalization and cable and conduit routing system according tothe present invention.

FIG. 4 illustrates illustrate a telecommunication bracket used in thethermal compartmentalization and cable and conduit routing systemaccording to the present invention.

FIG. 5A illustrates a top view of a data center or co-location facilityaccording to another embodiment of the present invention.

FIGS. 5B1 and 5B2 illustrate cut-away perspective views of an exteriorand interior portion of the data center or co-location facilityaccording to other embodiments of the present invention.

FIGS. 6A and 6B illustrate other telecommunication brackets used in thethermal compartmentalization and cable and conduit routing systemaccording to the present invention.

FIGS. 7A and 7B illustrate a section of a distribution area and the dataarea within a facility according to an embodiment of the presentinvention.

FIG. 8 illustrates a power spine that can also be used with thepreferred embodiment.

FIGS. 9A, 9B, 9C, 9D, and 9E illustrate an air handling unit accordingto a preferred embodiment.

FIG. 10 illustrates a control system used by the data center.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention provides data center or co-location facilitydesigns and methods of making and using the same. The data center orco-location facility designs have certain features that will be apparentherein and which allow many advantages in terms of efficient use ofspace, efficient modular structures that allow for efficiency in theset-up of co-location facility and the set-up of the electronicsequipment in the facility, as well as efficient air-conditioning withinthe facility. Each of these features has aspects that are distinct ontheir own, and combinations of these features also exist that are alsounique.

FIG. 1(a) illustrates a floor design used in a data center orco-location facility according to the present invention. The preferredembodiment discussed herein uses parallel rows of equipment configuredback-to back so that each row of equipment generally forces the heatfrom the electronic equipment towards a hot aisle, thus alsoestablishing a cold aisle in the front of the equipment. The cold aislesin FIG. 1(a) are illustrated at the dotted line block 60, wherein thehot aisles are illustrated at the dotted line block 62. One feature ofthe present invention is the provision for marking the floor 50 toexplicitly show the various areas of the facility. As illustrated, thehot aisle 62 has a central area 52 that is tiled, painted, taped orotherwise marked to indicate that it is center area of the hot aisle 62,also referred to as a central hot air area. The typical dimensions ofthe central area 52 are typically in the range of 2′-4′ across thewidth, with a row length corresponding to the number of electroniccabinets in the row. Marking with tiles is preferable as the markingwill last, and tiles that are red in color, corresponding to thegeneration of heat, have been found preferable. Around this center area52 is a perimeter area 54, over which the cabinets are installed. Thisperimeter area 54 is marked in another manner, such as using a grey tilethat is different in color from the center area 52. Around the perimeterarea 54 is an outside area 56, which is marked in yet a differentmanner, such as using a light grey tile. The placement of these markingsfor areas 52, 54 and 56 on the floor of the facility, preferably priorto moving any equipment onto the floor, allows for a visualcorrespondence on the floor of the various hot and cold aisles. Inparticular, when installing cabinets over the perimeter 54 are, the areathat is for the front of the cabinet that will face the cold aisle, andthus the area for the back of the cabinet for the hot aisle, is readilyapparent.

FIG. 1(b) illustrates floor-based components disposed over the floordesign of the co-location facility according to the present invention.FIG. 1(b) also shows additional area of the floor, which in thisembodiment is provided to illustrate interaction of the electronicsequipment with the evaporators of the air conditioning units. In theembodiment described with respect to FIG. 1(b), certain features areincluded so that conventional equipment, particularly conventional airconditioning equipment, can effectively be used while still creating thedesired air flow patterns of the present invention as described herein.

Before describing the components in FIG. 1(b), an aspect of the presentinvention is to isolate the hot air exhaust from the areas that requirecooling as much as possible, and to also create air flows in which theair moves through the exhaust system, into the air conditioning system,through the air conditioning ducts and out to the cool equipment in avery rapid manner. In particular, the amount of circulation establishedaccording to the present invention moves air at a volume such that theentire volume of air in the facility recirculates at least once every 10minutes, preferably once every 5 minutes, and for maximum cooling onceevery minute. It has been found that this amount of recirculation, incombination with the air flows established by the present invention,considerably reduce the temperature in the facility in anenvironmentally efficient manner.

Cabinets 110 shown in FIG. 1(b) are placed generally over the sides ofthe perimeter 54 as described, in rows. Different rows are thus shownwith cabinets 110(a-f), with each letter indicating a different row.Also included within the rows are telecommunications equipment 170 towhich the electronics equipment in each of the cabinets 110 connect asdescribed further herein, as well as power equipment 180, containingcircuit breakers as is known to protect against energy spikes and thelike, that is used to supply power along wires to the electronicsequipment in each of the cabinets 110 connect as described furtherherein. Air conditioning units include the evaporator units 120 (1-6)that are shown being physically separated by some type of barrier fromthe area 56 described previously with respect to FIG. 1(a). Thecondenser units of the air conditioning system that receive the warmedrefrigerant/water along lines 122 and are disposed outside the walls ofthe facility are not shown. This physical separation is implemented inorder to establish warm exhaust channel area 240 separate from thephysical space, which warm air area will connect to a separate warm airarea in the ceiling and allow the warm air to flow into the exhaustchannel area 240 and enter into intake ducts of evaporator airconditioning equipment 120, as will be described. This feature allowsthe usage of conventional evaporator air conditioning equipment that hasair intakes at the bottom of the unit, as well as allows for usage ofdifferent air conditioning equipment types, while still maintaining anefficient airflow throughout the entire facility.

FIG. 1(c) illustrates a perspective cut-away view along line c-c fromFIG. 1(a) of the FIG. 1(a) co-location facility according to the presentinvention. Additionally, illustrated are the false ceiling 140 and theactual ceiling 150, which have a gap that is preferably at least 1.5-3feet and advantageously at least 15 feet, as the higher the ceiling themore the warm air rises (and thus also stays further away from theequipment in the cabinets 110). The false ceiling 140 is preferably madeof tiles that can be inserted into a suspended ceiling as is known,which tiles preferably have are drywall vinyl tiles, which exhibit agreater mass than many conventional tiles. Also shown are arrows thatillustrate the air flow being centrally lifted upward from the hot airarea containment chamber 210 formed by the thermal shields 400 to thearea between the false ceiling 140 and the actual ceiling 150, and theflow within the ceiling toward the warm exhaust channel area 240, andthen downward into the warm exhaust channel area 240 with the wall 130separating the area 56 and the warm exhaust channel area 240. Also shownare arrows that take cold air from the cold air ducts 310 and insert theair into the cold aisles 60.

Though the arrows in the drawing are directed straight downward, thevents themselves can be adjusted to allow for directional downward flowat various angles. In a preferred embodiment, each of the vents have aremote controlled actuator that allows for the offsite control of thevents, both in terms of direction and volume of air let out of eachvent. This allows precise control such that if a particular area isrunning hot, more cold air can be directed thereto, and this can bedetected (using detectors not shown), and then adjusted for offsite.

FIGS. 2(a)-(c) illustrate various cut-away perspective views of thethermal compartmentalization and cable and conduit routing systemaccording to the present invention. In particular, FIG. 2(a) illustratesa cut away view of a portion of the hot air area containment chamber210, which rests on top of the cabinets 110, and is formed of aplurality of the thermal shields 400 and 450, which are modular inconstruction and will be described further hereinafter. Also illustratedare shield brackets 500 that are mounted on top of the cabinets 110, andprovide for the mounting of the shields 400 and 450, as well as an areaon top of the cabinets 110 to run power and telecommunications cables,as will be described further herein.

Before describing the cabling, FIG. 2(b) and FIG. 4 illustrate theshield and cabling support bracket 500, which is made of structurallysound materials, such as steel with a welded construction of the variousparts as described, molded plastic, or other materials. Ladder racksupports 510, 520, 530, 540 and 550 are attached to back verticalsupport 502 of the shield and cabling support bracket 500 and used toallow ladder racks 610, 620, 630, 640, and 650 respectively, placedthereover as shown. The ladder racks are intended to allow for asegregation of data and electrical power, and therefore an easier timenot only during assembly, but subsequent repair. The ladder racks areattached to the ladder rack supports using support straps shown in FIG.4 , which are typically a standard “j” hook or a variant thereof. Asalso illustrated in FIG. 4 , a support beams structure 506 providesextra support to the ladder rack, and the holes 508 are used to securethe shields 400 and 450 thereto. Horizontal support plate 504 is used tosupport the support bracket 500 on the cabinets 110.

With respect to the cabling and conduit, these are used to provideelectrical power and data to the various servers in the facility.Conduit, containing wiring therein, is used to provide electricity.Cabling is used to provide data. In this system, it is preferable tokeep the electrical power and the data signals separated.

Within the system, ladder rack 610 is used for data cabling on the coldaisle side of the thermal shields 400. Ladder rack 620 is used for anA-source power conduit (for distribution of 110-480 volt power) on thecold aisle side of the thermal shields 400. Ladder rack 630 is used forB-source power conduit (for distribution of 110-480 volt power), whichis preferably entirely independent of A-source power conduit, on thecold aisle side of the thermal shields 400. Ladder rack 640 is used formiscellaneous cabling on the cold aisle side of the thermal shields 400.Ladder rack 650 is used for data cabling on the hot aisle side of thethermal shields 400.

FIGS. 3(a) and (b) illustrate modular thermal shields 400 and 450,respectively, used in the thermal compartmentalization and cabling andconduit routing system according to the present invention. Both shields400 and 450 are made of a structurally sound material, including but notlimited to steel, a composite, or a plastic, and if a plastic, one thatpreferably has an air space between a front piece of plastic and a backpiece of plastic for an individual shield 400. Shield 400 includes athrough-hole 410 that allows for certain cabling, if needed, to runbetween the hot and cold aisle areas, through the shield 400. Athrough-hole cover (not shown) is preferably used to substantially closethe hole to prevent airflow therethrough. Shield 450 has a 90 degreeangle that allows the fabrication of corners.

It should be appreciated that the construction of the cabinets, theshields 400 and 450, and the shield supports 500 are all uniform andmodular, which allows for the efficient set-up of the facility, as wellas efficient repairs if needed.

Other different embodiments of data center or co-location facilitiesaccording to the present invention also exist. For example, while thefalse ceiling 140 is preferred, many advantageous aspects of the presentinvention can be achieved without it, though its presence substantiallyimproves airflow. Furthermore, the evaporation units for the airconditioning system can also be located outside the facility, in whichcase the chamber 240 is not needed, but hot air from the ceiling can bedelivered to evaporation units that are disposed above the ceiling,which is more efficient in that it allows the warm air to rise. If thecomplete air conditioning equipment is located outside, including theevaporators, the refrigerant/water lines 122 that are used to exchangethe refrigerant/water if the evaporators are disposed inside thefacility is not needed, which provides another degree of safety to theequipment therein.

It is noted that aspects of the present invention described herein canbe implemented when renovating an existing facility, and as such not allof the features of the present invention are necessarily used.

Data Management Center and Integrated Wiring System

In one aspect, the embodiments herein are directed to an overall datamanagement center, including the building itself, interior aspects ofthe building, as well as equipment purposefully located outside yet inclose proximity to the building, which equipment is used for purposes ofproviding both building cooling as well as supplemental power, asdescribed further herein. In one particular aspect, the center core ofthe building that contains the electronics equipment is purposefullycreated in a manner that provides only essential equipment and ductsneeded to provide power, communications, and air flow, while puttinginto periphery areas of the building and outside, other equipment thatcould interfere with the electronics equipment, whether due to thatother equipment requiring extremely high power and/or water or otherliquids to function, all of which can have a detrimental impact on theelectronics equipment.

FIG. 5(a) illustrates a top view of a portion of a data center orco-location facility 580 according to another embodiment of the presentinvention. In this embodiment, unlike the embodiment shown in FIG.1(a)-(c), the condenser air conditioning units 800 and heat expulsionchamber 900 are all disposed outside of the exterior walls 582 of thefacility, as will be described further herein. There is also additionalequipment disposed outside of the exterior walls 582, includingevaporation units 591 that feed cooled water along lines 592 to the airconditioning units 800 as described further herein, as well as backupdiesel generators 594 for supplying backup power along a transmissionline 596 in the case of power outage from remotely supplied power on thenational power grid.

FIG. 5(b) 1 illustrates a cut-away perspective view of an exterior andinterior portion (with a 90° rotation for illustrative purposes of theinterior portion) of the data center or co-location facility 580, withthe exterior wall 582 being explicitly illustrated. Shown are two of thecabinet clusters 590-1A and 590-2A, and the corresponding hot air areacontainment chambers 210 and cold air ducts 310, which are respectivelyconnected to the warm exhaust outlets 240-0 and cold duct inlets 310-I.The warm exhaust outlets 240-0 and cold duct inlets 310-I connect toheat expulsion chamber 900 and condenser units 800, respectively.

FIG. 5(b) 2 provides a slightly varied embodiment, in which the coldduct inlets 310-I and warm exhaust outlets 240-0 are each at the samelevel as the condenser units 800 and heat expulsion chamber 900,respectively, and the warm exhaust outlets 240-0 contain a 90° angledarea, which allows for better hot air flow into the heat expulsionchambers 900.

Within the facility there are provided distribution areas 584 and 588,as shown in FIG. 5(a), as well as data center equipment areas 586, whichequipment areas 586 each contain an array of cabinet clusters 590 (shownin one of the rows as cabinet clusters 590-1, 590-2, 590-3 . . . 590-N),since within each cabinet cluster 590, various cabinets 110 containingdifferent electronic equipment are disposed in rows, thereby allowingeach cabinet cluster 590 to be locked, as well as the cabinets 110within the cabinet cluster 590. It is apparent that three consecutivecabinet clusters, such as 590-1, 590-2 and 590-3 correspond to the threeidentified clusters that are disposed around the associated hot air areacontainment chambers 210(a), 210(b) and 210(c) in FIG. 1(B). As isillustrated, the electronics equipment within each cabinet 110 of acabinet cluster 590 is connected in a manner similar to that asdescribed in FIGS. 2(a)-(c) previously.

It is noted that the cabinet cluster may have an actual physicalperimeter, such as a cage built with fencing that can be locked andstill permits airflow therethrough, or alternatively need not have anactual physical perimeter, in which case the orientation of the cabinets110 and corresponding other structures as described previously withreference to FIGS. 1 (a-c) can also define this same space.

The manner in which the distribution power wires and conduits,electronic equipment control wires and conduit, data cabling, andmiscellaneous cabling is distributed to the cabinet clusters 590 fromone of the distribution areas 584 or 588 will be described furtherhereinafter. As shown in FIG. 5(a), telecommunications and powerdistribution equipment, further described herein, is used to then feedthe appropriate signals and power to the telecommunications equipmentand power equipment that is stored within each cabinet cluster 590 (i.e.telecommunications equipment 170 and power equipment 180 described inFIG. 1(B)). The manner in which the distribution power wires andconduits, electronic equipment control wires and conduit, data cabling,and miscellaneous cabling is distributed to the cabinet clusters 590from one of the distribution areas 584 and 588 will be described furtherhereinafter.

The array of cabinet clusters 590, and the density of the cabinets 110and the electronics equipment therein, require substantial amounts ofpower and transmission capacity, which in turns requires substantialamounts of wiring, particularly for power. As described herein, as aresult there is described an improved telecommunication bracket 600,which substantially rests over each of the cabinets in the cabinetclusters 590, in order to more easily accommodate the distribution powerwires and conduits, as well as telecommunication wires and conduits, aswell as control wires and conduits, that are then distributed from thedistribution areas 584 and 588 to the telecommunications equipment 170and power equipment 180 that is within each of the different cabinetclusters 590. As shown in FIG. 5A and FIG. 8 , the distribution area 588contains PDU's 598, described in further detail elsewhere herein, andthe distribution area 584 contains transformers to step down the powergrid power that is normally at 12477 volts to a 480 volt level, fortransmission of 480 volt power to the PDU's 598. Also withindistribution area 584 are uninterruptable power supplies in case anoutage of power from the power grid occurs, as well as equipment fortesting of the various power equipment that is conventionally known.

While FIG. 1B illustrates one configuration of equipment with thecabinet cluster (with the telecommunications equipment 170 and the powerequipment 180 within the center of a row), FIG. 7A also shows analternative configuration of equipment for a cabinet cluster 590, whichstill contains the same cabinets 110, telecommunication equipment 170and power equipment 180. In particular, rather than having the powerequipment 180 centrally located within a row, in this alternateconfiguration the power equipment 180 is disposed at an end of each ofthe rows that are within a cabinet cluster 590. The telecommunicationequipment, within this embodiment, can be located anywhere within therow of cabinets 110, within whichever one of the cabinets 110 makes mostsense given the usage considerations for that cabinet cluster 590.

In another variation of the FIG. 7A embodiment, the power equipment 180,instead of being somewhat separated from the cabinets 110 within acluster 590, instead abut right next to one of the cabinets 110. This,along with the doors 593 shown in FIG. 7A then being attached betweenadjacent power equipment at the end of the cabinet row instead of at theend cabinet, keep all the equipment in a tightly configured space. Inany of the embodiments shown, whether FIG. 1C, 7A or as described above,the thermal shield 400 that creates the hot air area containment chamber210 above the cabinets, coupled with the doors that seal off the areabetween the rows of cabinets 110 within a cluster 590, provide anenvironment that prevents the hot air within the hot air area 52 fromescaping out into the main data center floor, and ensures that the hotair instead travels up through the hot air area containment chamber 210and into the gap disposed between the false ceiling 140 and the actualceiling 150.

Within equipment area 586 is thus established an array of cabinetclusters 590, which cabinet clusters align with each other to allow forthe overhead stringing of telecommunications and power wiring asdescribed herein. Within each cabinet cluster 590, as also shown in FIG.1B, is telecommunications equipment 170 to which the electronicsequipment in each of the cabinets 110 connect, as well as powerequipment 180 used to connect the electronics equipment to power. Thearray of cabinet clusters 590, each also containing brackets, such asbrackets 500 or 600, as described herein. For a larger size data centeras illustrated in FIG. 5(a) that contains a very large array of cabinetclusters 590, brackets 600 are preferable, as they allow for additionalconduit support areas. These brackets 600, discussed further herein withrespect to FIGS. 6 (a-b), contain ladder racks 510, 520, 530 and 540that are used for stringing power and telecommunication wiring withineach cabinet cluster 590, as well as contain additional vertical supportwith conduit clamps that are used to hold power and telecommunicationlines that pass from each cabinet cluster 590 to other centraltelecommunication and power distribution areas, as discussed furtherherein, as well as to hold power and telecommunication lines that passover certain of the cabinet clusters 590 in order to be strung to otherof the cages areas 590. Still further, these same brackets 600, beingpreferably mounted over the cabinets 110, and at least having asignificant portion of the bracket disposed over the cabinets 110, areused to mount the thermal shield within the cabinet cluster 590, thethermal shield providing a contiguous wall around the central hot airarea of the cabinet cluster 590, and defining a warm exhaust channelthat traps the heated air within the central hot air area and causessubstantially all the heated air within the central hot air area to riseup within the warm exhaust channel. These brackets 600 also preferablyspan from the top of the cabinets 110 to the bottom of the false ceiling140 to provide further stability.

It is apparent that the power and telecommunication lines that pass fromeach cabinet cluster 590 to other more central telecommunication andpower distribution areas will necessarily pass, in some instances, overother cabinet clusters 590. Since the vertical support 610 with conduitclamps 620 are above the ladder racks 510, 520, 530 and 540 for each ofthe brackets 600, as well as above each of the cabinets 110, this allowsfor long runs of power and telecommunication lines that pass from eachcabinet cluster 590 to other more central telecommunication and powerdistribution areas to exist without interfering with the wiring thatexists within each cabinet cluster 590. Furthermore, by creating asufficient area of vertical support and conduit clamps, it is thenpossible to run additional power and telecommunication lines fromcertain cabinet clusters 590 to other more central telecommunication andpower distribution areas without having to re-work existing wiring. Thismakes expansion much simpler than in conventional designs.

FIGS. 6 a-b illustrate in detail two different embodiments of thetelecommunication bracket 600 referred to above that is used in thethermal compartmentalization and cable and conduit routing systemaccording to the present invention. This bracket 600 serves the samepurpose as the bracket 500 illustrated and described previously withrespect to FIG. 4 , and as such similar parts of the bracket 600 arelabeled the same and need not be further described herein. This bracket600, however, additionally provides additional vertical support 610 thatallows for the running of additional wiring and conduits.

In FIG. 6A, this additional vertical support 610 includes conduit clamps620 that allow the clamping of the additional conduits to the additionalvertical support 610.

In FIG. 6B, the bracket 600A has in addition to the vertical support 610a support beam 506A (which extends upwards from the support beam 506shown in FIG. 4 ), and racks 630, 632, 634, 636, 638, and 640therebetween. Each of the racks 630, 632, 634, 636, 638, and 640 haveroom for at least 4 different 4″ conduits to run wiring or cablingtherethrough. Whether the conduit clamps or additional conduit racks areused, both provide for conduit holding, and holding of the wires orcables within the conduits.

In both the brackets 600 of FIGS. 6 a-b , the additional wiring/conduitis distribution power wires and conduits and other wire/conduit forcontrol uses, for example. As explained hereafter, the distributionpower wires and conduits can run from various power equipment units 180disposed in each of the cabinet clusters 590 to various other high powerdistribution units (PDUs) 598 disposed within the distribution area 588,as shown in FIG. 7A.

FIG. 7A also illustrates the distribution of power PDUs 598 within asection of the distribution area 588 to power equipment 180 in an endcabinet cluster 590-1 within a section of the data equipment center area586 via distribution power wires and conduit (one shown as 597). Inparticular, as is shown, distribution power wires and conduit goes fromeach of the PDUs 598A and 598B to the power equipment unit 180A withinthe end cabinet cluster 590-1, and distribution power wires and conduitalso goes from both the PDUs 598A and 598B to the power equipment unit180B within the end cabinet cluster 590-1, so that redundant power canbe provided to the electronic equipment within each row. Since power isprovided to each piece of power equipment 180 from two differentsources, these power equipment units can also be called redundant powerpanels, or RPP's. In addition, distribution power wires and conduit gofrom each of PDUs 598A and 598B over the end cabinet cluster 590-1 tofurther cabinet clusters 590-2, 590-3 to 590-N. The array of cabinetclusters 590 are aligned as shown in FIG. 5(a) so that the brackets 600in different cabinet clusters 590 nonetheless can together be used tostring distribution power wires and conduit and other wires/fibers withconduits as needed.

In a preferred configuration of the power equipment 180 shown in FIG. 7Aprovides redundant 120 volt AC power from each RPP 180 to the electricalequipment in each of the cabinets 110 within the row of the cabinetcluster 590. Within the RPP 180 are circuit breakers as is known toprotect against energy spikes and the like, as well as energy sensorsassociated with each circuit so that a central control system, describedhereinafter, can monitor the energy usage at a per circuit level. In atypical implementation, there are 42 slot breaker panels that areassociated each with 120 c/208 v power that is then supplied to each ofthe electronic components as needed, in wiring that uses one of theladder racks 630 or 640 as discussed previously to the necessary cabinet110. Of course, other power configuration schemes are possible as well.

In a preferred configuration for a module of cabinet clusters 590, asschematically shown in FIG. 7B, there are three different PDUs 598 thateach receive 480 vAC 3-phase power and provide 120 vAC 3-phase powerservice to each of 8 different RPPs 180 via the distribution power wiresand conduits. This allows, for a completely used module, 6 differentcabinet clusters 590 to be serviced from 12 RPP's 180, two in each cage,and 3 different PDU's 598. By providing redundancy of both RPP's 180(x2) and PDUs 598 (x3), this allows for maximum power usage of thevarious components with sufficient redundancy in case any one of thePDU's 598 or any circuit on an RPP 180 fails.

A lock-related aspect of the present invention with respect to the RPPs180 as well as the PDU's 598 is that since there are three circuits fromthe PDU's t the RPP's, within a dual RPP each side of the cabinet willhave separate lock, such that all locks of a particular circuit can beopened by the same key, but that key cannot open locks of any of theother two circuits. This is an advantageous protection mechanism, as itprohibits a technician from mistakenly opening and operating upon adifferent circuit than a circuit he is supposed to service at that time.

FIG. 8 shows a power spine 599 that can also be used with the preferredembodiment to provide power from the power grid to each of the PDU's598. As illustrated, rather than running the power spine through theroof as is conventionally done, in this embodiment the power spine 599is run along a corridor within the distribution area 588 that channelsall of the main building wiring and electrical components. Thisadvantageously reduces stress on the roof and building structure, as theweight of the power spine and related components are supportedinternally within the corridor structure as shown.

Data Center Air Handling Unit

Another aspect of the data center is the air handling unit that providesfor efficient cooling.

As is illustrated in FIGS. 5(a) and 5(b) 1-2, one condenser unit 800 ispaired with one heat expulsion chamber 900, and each are preferablyindependently movable. As is further illustrated, the condenser units800 are built to a size standard that allows for transport along USstate and interstate highways. Further, the heat expulsion chamber 900is preferably sized smaller than the condenser unit 800, but stillhaving dimensions that allow for transport using a semi-trailer. Whentransported to the facility 500, the condenser unit 800 is first placedinto position, as shown here on posts 588, but other platforms can alsobe used. As shown in this embodiment, the heat expulsion chamber unit900 is placed over the condenser unit 800, though other placements, suchas adjacent or below, are also possible. Connections of power conduit,miscellaneous cabling, and water needed for proper operation of thecondenser units 800 and expulsion chamber 900 is preferably made usingeasily attachable and detachable components.

With this configuration, the units 800 and 900 are located instandardized, accessible and relatively convenient positions relative tothe facility 580 should any of the units 800/900 need to be accessedand/or removed for repair or replacement. Further, these units 800/900are themselves created using an intentionally transportable design.

FIGS. 9A-9E provide further details regarding the condenser unit 800 andits paired heat expulsion chamber 900. In particular, as shown, the airconditioning apparatus includes the condenser unit 800 and its pairedheat expulsion chamber 900. The heat expulsion chamber 900 receivesheated air, and emits vented air, and the vented air is released intothe external environment, while the condenser unit 800 emits cooled air.

The heat exchange unit 900 contains an exhaust fan 910, controlled by aVFD fan control and I/O signals block 1330 shown in FIG. 10 , that emitsheat from the heated air as the vented air, thereby allowing return airto pass through a return damper 920, which return damper 920 has areturn damper actuator associated therewith.

The condenser unit 800 includes an outside air inlet 810, and hasassociated an outside air damper 812, thereby allowing outside air topass therein. This outside air damper 812 is preferably coated with aneoprene seal to prevent pollution particles from passing through thedamper 812 when in a closed position, as well as contains aspring-loaded mechanism closing lever that will automatically close theoutside air damper 812 upon a removal of power, so that outside air isprevented from intake before backup generators 594 have to start, sinceafter a power-grid power failure condition, before the back-upgenerators start, uninterruptable power supplies will supply buildingpower, giving a period for the outside air damper 812 to close.

A filter chamber 820, which includes an air intake area 822 coupled tothe heat expulsion unit 900 and the outside air inlet 810, isconfigurable, via the AHU control system 1000, described hereinafter, toreceive the return air, the outside air, as well as a mixture of thereturn air and the outside air, the filter chamber resulting in filteredair. In a preferred implementation of the filters 824 within the filterchamber 820 are included a MERV 7 screen filter 824A with a MERV 16 bagfilter 824B therebehind, which allows replacement of the screen filter824A without replacement of the bag filter 824B, and vice-versa.

The condenser unit 800 includes an air cooling area 830 over which thefiltered air passes to create the cooled air. For ease of nomenclature,all of the air within the air cooling area 830 is referred to asfiltered air, and only upon emission from the condenser unit is itreferred to as cooled air. That notwithstanding, it is understood thatalong various stages of the air cooling area 830, the filtered air willget progressively cooler in temperature.

The air cooling area 830 of the condenser unit 800 includes a directcooling coil 840 filled with a gas for direct expansion, such as R134gas, over which the filtered air passes, the gas being circulatedthrough a condenser 842 disposed in another area of the condenser unithousing, but still in the external area, outside of the building.

The air cooling area 830 also includes an indirect cooling coil 850filled with cooled water over which the filtered air passes, the cooledwater being circulated through an evaporation unit 590 also disposed inthe external area, via a water line 592 as shown in FIG. 5(a).Optionally, though not shown, another coil that is cooled by a chillercould be included.

Also shown in FIGS. 9A-9E is that the air cooling area also has anevaporator 860 that provides a water wall through which the filtered aircan pass. An evaporator bypass 862 allows all or some of the filteredair to bypass the evaporator 860, and a bypass damper 880 is opened toallow 100% bypass of the evaporator 860, in which case the evaporatordamper 890 is then fully closed. Filtered air can also be partiallybypassed, or all go through the evaporator 860, depending on thepercentage opening of each of the dampers 880 and 890.

Also within the air cooling area 830 is a fan 870, shown as a fan arrayof multiple fans, operable to push the filtered air through the aircooling area 830, as well as an outlet damper 880 controllable by anactuator and operable to control an amount of the cooled air deliveredfrom the air cooling area 830.

As shown and mentioned previously the heat exchange unit 900 iscontained within a first housing, and the condenser unit 800 iscontained within a second housing.

Furthermore, and with reference to FIG. 10 , overall air conditioningsystem for the data center 500 includes a control system 1000. Thecontrol system 1000 contains an air handling unit (AHU) and powercontrol system computer 1100, which is operable to automatically controleach of the exhaust fan 910, the return damper actuator, the outside airdamper actuator, the condenser 842, the bypass damper actuator, the fan870, and the outlet damper actuator.

Air Handling Control System

As referenced previously, and shown explicitly in FIG. 10 , the datacenter 580 includes a control system 1000. The control system includesan air handling unit (AHU) and power control system (PCS) computer 1100,which as shown obtains signals from many different units, and sendssignals to many different units, based upon various software routinesrun by the AHU/PCS computer 1100. These routines can be integrated witheach other, as well as be discrete modules which operate on their own,or a combination of both.

A significant aspect of the present invention is the placement ofsensors that can monitor for each/all of temperature, pressuredifferential, airflow, and humidity. Sensors that monitor thesedifferent aspects are placed in different locations throughout the datacenter.

In particular, having temperature sensors inside the thermal shield 400(preferably redundant ones at the two ends and the middle of the clusterat least), and at different levels (such as at the middle and top of acabinet 110, as well as at the middle and top of the thermal shield400), as well as in stratified locations in the gap between the falseceiling 140 and the actual ceiling 150 (spaced at intervals of between2-4 feet, as well as outside the thermal shield area, at the outside ofcabinets in the cold aisles, allows for precise temperature gradientinformation throughout the facility.

Humidity sensors are helpful to have at locations that are the same asthe temperature sensors, though fewer are needed, as humidity data neednot be as precise for overall control of the building thermalenvironment.

Pressure differential sensors are also preferably located, redundantly,in a number of different areas. These include within the thermal shieldbelow the false ceiling 140, outside the thermal shield below the falseceiling 140, at different locations in the gap between the false ceiling140 and the actual ceiling 150 (spaced at intervals of between 2-4feet), at various locations within the cold aisle ducts 310,particularly a header plenum that has a main cold air area to which manyof the different condenser units connect, shown best along 310-I in FIG.5B2 and then distribute cool air to the cooling ducts 310 that form thecold aisles. This allows for sensing of the pressure at variouslocations, and in particular within the hot air containment chamber 210,outside the hot air containment chamber 210 above the cabinets 110,within the gap between the false ceiling and the actual ceiling 150, andwithin the cold aisle ducts. This allows for modification of the airhanding units 800/900 by the control system 1100. Overall pressurecontrol between the hot air containment chamber 210, the cold aisle, andthe gap between the false ceiling and the actual ceiling 150 is achievedby adjusting the air handling units 800/900 so that the pressure ismaintained in these different areas within a predetermined range of eachother, for example. This also allows for running the facility at apositive pressure differential when outside air is used, at ranges of 1%to 6%, such that as in essence the building breathes out.

Airflow sensors are also preferably located in each of the areas wherethe pressure differential sensors are noted as being required, in orderto ensure that the airflow is stable, as amounts of airflow that are toogreat, just as pressure differentials that are too great, can adverselyaffect the electronic equipment.

Areas where these differentials occur the most in the embodimentsdescribed herein are at the barrier caused by the thermal shield 400within each cabinet cluster 590, between the false ceiling and the gapthereover, since heated air from each of the different hot aisle areas210, associated with each cabinet cluster 590, vent to this large gaparea.

Signals from these sensors, as shown by Temperature, PressureDifferential, Airflow, and Humidity Sensor Control and input/output(I/O) signals (block 1310) can then be used to provide damper actuatorcontrol (block 1320), VFD fan control and I/O signals (block 1330),evaporator control and I/O signals (block 1340), condenser control andI/O signals (block 1350), evaporator control and I/O signals (block1360), and optionally chiller control and I/O signals (block 1370).Within the Damper actuator control block is included the dampersassociated with the cold aisle ducts, which dampers can be automaticallyadjusted to fully open, fully closed, or in-between amounts based uponsensing of the current conditions, as described previously.

Still furthermore, the AHU/PCS computer 1100 also monitors powerconsumption and power production, depending on the devices, to assessoverall power usage. As such, electrical energy monitor sensors withinthe RPP 180 are operated upon by the RPP control and I/O signals block1410, and provide an indication of the power usage of the electronicsdevices in the cabinets 110. The PDU 598 is monitored, as is known, andoperated upon by the PDU control and I/O signals block 1420. Power loadcontrol and I/O signals block 1430 provides monitoring of thetransformers and uninterruptable power supplies within the distributionarea 584. Backup generator control and I/O signals block 1440 is usedfor the control of the backup generator 594, whereas telecommunicationcontrol and I/O signals block 1450 is used for the control of thetelecommunications equipment. Equipment load control and I/O signalsblock 1460 controls and monitors energy consumption of other equipmentwithin the data center facility

The above control blocks can contain software written to both act uponinput signals obtained from other sensors or other units, and ensurethat the various different units operate together. The usage of the termI/O signals is intended to convey that for any of the associatedsensors, actuators for dampers, VFD for fans, and other mechanisms, thatdepending on the model used, such devices may output signals, inputsignals or both.

It is also noted that what occurs with one device will alter which otherdevices operate. Thus, for example malfunction of a particular circuitin an RPP 180 will cause the AHU/PCS computer 1100 to switch over to theredundant circuit in the same RPP 180 until that circuit is fixed.

It is particularly noted that the above system can monitor and controlfor certain situations that are particularly significant for datacenters. For example, the air flow patterns that are caused, with theinclusion of the false ceiling 140 as shown in FIG. 1C, requireassessment of high and low pressure areas. The AHU/PCS computer 1100 canmonitor for this, and as a result maintain a balance, thus ensuring thatfans and other components that are within the electronics equipmentstored in the cabinets 110 isn't damaged.

Also shown in FIG. 10 are building cabinet cluster and cage lock sensorsblock 1510. This allows for the detection of which cabinet clusters 590,as well as which cabinets 110, are open, based upon sensors that areplaced at each of these areas.

Fire and roof water detection leak sensors module 1520 is also shown, asthis can be used in conjunction with known systems, and interfaced withthe other blocks referred to herein, to ensure that if a fire or leak isdetected, that appropriate shut down of equipment in the preferredsequence to avoid damage is done.

Although the present invention has been particularly described withreference to embodiments thereof, it should be readily apparent to thoseof ordinary skill in the art that various changes, modifications andsubstitutes are intended within the form and details thereof, withoutdeparting from the spirit and scope of the invention. Accordingly, itwill be appreciated that in numerous instances some features of theinvention will be employed without a corresponding use of otherfeatures. Further, those skilled in the art will understand thatvariations can be made in the number and arrangement of componentsillustrated in the above figures.

What is claimed is:
 1. A facility with an internal area separate from anoutdoor environment, the internal area for maintaining electronicequipment disposed in a plurality of cabinet clusters, the facilitycomprising: a building that includes an exterior load wall separatingthe internal area from the outdoor environment, the internal areacontaining a volume of air; one or more wall openings in the exteriorload wall; a floor within the internal area on which the plurality ofcabinet clusters are disposed; a plurality of cabinets for holding theelectronic equipment therein, the plurality of cabinets forming aplurality of rows, the plurality of rows forming the cabinet clusters,the cabinets arranged so that the electronic equipment disposed withinthe cabinets emit heated air from the cabinets; an air conditioningsystem comprising a plurality of evaporator units and a plurality ofcondenser units, the plurality of condenser units being located in theoutdoor environment, the plurality of evaporator units being configuredto receive the heated air, condition the heated air to create cooledair, and emit the cooled air back into the internal area; a warm airescape channel, within the building but separate from the cabinetclusters, that extends through the wall openings that is configured todirect the heated air from the cabinets, through the wall openings tothe plurality of evaporator units, which are disposed to receive theheated air from the warm air escape channel; and a cool air pathwaywithin the building extending from the plurality of evaporator unitsthrough the wall openings and towards the cabinet clusters to providethe cooled air to a cold aisle, wherein the cool air pathway comprises aplurality of air ducts structured to output the cooled air adjacent aceiling of the cold aisle, the ceiling being disposed opposite thefloor.
 2. The facility according to claim 1 wherein the one or more wallopenings in the exterior load wall are configured to attach to one endof a warm air escape channel connector or a cool air pathway connector.3. The facility according to claim 2 wherein another end of the warm airescape channel connector or cool air pathway connector is configured toattach to the plurality of evaporator units.
 4. The facility accordingto claim 1 wherein the air conditioning system also receives outdoorair.
 5. The facility according to claim 1 wherein the air conditioningsystem further includes an outdoor air inlet that allows intake ofoutside air to a filter chamber, and a spring-loaded mechanical closinglever, the spring-loaded mechanical closing lever causing automaticclosure of an outside air damper disposed within the outside air inletupon a disruption in electrical power, thereby preventing the intake ofthe outside air that contains gaseous exhaust to the filter chamber uponthe disruption in electrical power.
 6. The facility according to claim 1wherein the heated air emitted from the cabinets is directed toward acentral hot air area associated with each cabinet cluster.
 7. Thefacility according to claim 6 further comprising a thermal shieldproviding a contiguous wall around a hot air containment chamber, thatis located above the central hot air area, the thermal shield traps theheated air within the hot air containment chamber and causessubstantially all the heated air within the central hot air area to becontained as it rises up from the central hot air area.
 8. The facilityaccording to claim 7 wherein the heated air rises up from within the hotair containment chamber into the warm air escape channel.
 9. Thefacility according to claim 7 wherein the warm air escape channel isdisposed within the building above the hot air containment chamber, thewarm air escape channel being lowerly bounded by a false ceiling. 10.The facility according to claim 1 wherein the plurality of evaporatorunits are disposed on a ground level support structure.
 11. The facilityaccording to claim 1 wherein the air conditioning system recirculatesthe volume of air at least once every 10 minutes.
 12. The facilityaccording to claim 1 wherein the air conditioning system recirculatesthe volume of air at least once every 5 minutes.
 13. The facilityaccording to claim 1 wherein the air conditioning system recirculatesthe volume of air at least once every minute.
 14. A facility with aninternal area separate from an outdoor environment, the internal areaconfigured to house electronic equipment disposed in a plurality ofcabinet clusters, the facility comprising: a building that includes anexterior load wall separating the internal area from the outdoorenvironment, the internal area containing a volume of air; one or morewall openings in the exterior load wall; a floor within the internalarea on which the plurality of cabinet clusters are disposed; aplurality of cabinets for holding the electronic equipment therein, theplurality of cabinets arranged into a plurality of rows of cabinets, theplurality of rows of cabinets forming a cabinet cluster so that theelectronic equipment disposed within the cabinets emits heated air fromthe cabinets; a thermal shield formed above the plurality of cabinets todirect the heated air into at least one thermal compartment; an airconditioning system comprising a plurality of evaporator units and aplurality of condenser units, the plurality of condenser units beinglocated in the outdoor environment, the plurality of evaporator unitsbeing configured to receive the heated air, condition the heated air tocreate cooled air, and then emit the cooled air back into the internalarea; a ceiling; a false ceiling located beneath the ceiling such that awarm air escape channel is formed between the ceiling and the falseceiling, so that the heated air is lifted upward from the at least onethermal compartment to the warm air escape channel and toward the one ormore exterior wall openings and into the plurality of evaporator units,which are disposed at an end of the warm air escape channel to receivethe heated air; and a cool air pathway within the building, wherein theplurality of evaporator units deliver the cooled air through the one ormore wall openings into the cool air pathway, the cool air pathwayconfigured to direct the cooled air towards cold aisles of the pluralityof rows of cabinets, wherein the cool air pathway comprises a pluralityof air ducts structured to output the cooled air near the ceiling of thecold aisles, the ceiling being disposed opposite the floor.
 15. Thefacility according to claim 14 wherein the one or more wall openings inthe exterior load wall are configured to attach to one end of a warm airescape channel connector or a cool air pathway connector.
 16. Thefacility according to claim 15 wherein another end of the warm airescape channel connector or cool air pathway connector is configured toattach to the air conditioning system.
 17. The facility according toclaim 14 wherein the air conditioning system further receives outdoorair.
 18. The facility according to claim 14 wherein the air conditioningsystem recirculates the volume of air at least once every 10 minutes.19. The facility according to claim 14 wherein the air conditioningsystem recirculates the volume of air at least once every 5 minutes. 20.The facility according to claim 14 wherein the air conditioning systemrecirculates the volume of air at least once every minute.